Predictive risk modeling for child maltreatment detection and enhanced decision-making: Evidence from Danish administrative data

Child maltreatment is a widespread problem with significant costs for both victims and society. In this retrospective cohort study, we develop predictive risk models using Danish administrative data to predict removal decisions among referred children and assess the effectiveness of caseworkers in identifying children at risk of maltreatment. The study analyzes 195,639 referrals involving 102,309 children Danish Child Protection Services received from April 2016 to December 2017. We implement four machine learning models of increasing complexity, incorporating extensive background information on each child and their family. Our best-performing model exhibits robust predictive power, with an AUC-ROC score exceeding 87%, indicating its ability to consistently rank referred children based on their likelihood of being removed. Additionally, we find strong positive correlations between the model’s predictions and various adverse child outcomes, such as crime, physical and mental health issues, and school absenteeism. Furthermore, we demonstrate that predictive risk models can enhance caseworkers’ decision-making processes by reducing classification errors and identifying at-risk children at an earlier stage, enabling timely interventions and potentially improving outcomes for vulnerable children.


B AUCs for different population groups
In this section, we provide summaries of the predictive performance of the models for different subsets of the full set of referrals.Table B.1 shows the performance of the models for boys and girls.In Table B.2, we show the results for children of non-Western and Western origin, respectively.Table B.3 shows the predictive performance of the models for children of a low-and high socioeconomic status (SES).Finally, Table B.4 presents the results by age groups.

C Additional external validation results
In this section, we provide additional evidence for the appropriateness of using child removal as an indicator of child maltreatment.To this end, we show in Table C.1 the prevalence of twelve additional adverse outcomes measured in 2017 for the three groups considered in Table 3:     Notes: In this table, we report the average characteristics of the individuals in the test sample with respect to their treatment status.The data is based on information in the year before the referral time.We define a CPS mistake as a referral for which no intervention (preventive service or removal) was initiated by CPS within the first four months, but where an intervention was initiated in the following eight months.Notes: In this table, we report the average characteristics of the individuals in the test sample with respect to their treatment status.The data is based on information in the year before the referral time.The risk scores in this table have been age-corrected.We define a CPS mistake as a referral for which no intervention was initiated by CPS within the first four months, but where an intervention was initiated in the following eight months.

Limited model
(i) the entire population of children residing in Denmark as of January 1, 2017 who were not involved in any referral during 2017; (ii) children involved in a referral in 2017 without a subsequent removal; and (iii) children involved in a referral in 2017 with a subsequent removal.In Fig C.1, we show the average values of these outcomes measured in 2018 as a function of the risk scores generated by the XGBoost model.In doing so, we focus on the 30% of the sample that constitutes the test sample.

Fig C. 1 .
Fig C.1.This figure shows the relationship between the predictions of the XGBoost models with the full information set and the risk of adverse child outcomes.Only the individuals in the test sample are used for this analysis.The blue line represents all referrals in the test sample, whereas the red line corresponds to the set of referrals in which CPS implemented no interventions in the first four months.The odds ratios for the binary outcomes, compare the odds of the outcomes for risk scores 9 and 10 (high-risk cases) relative to the odds for risk scores 1 and 2 (low-risk cases) using all referrals.Figures (g)-(k) are calculated only for children of compulsory school age (6-16) and are standardized to have zero mean and unit variance at the population level.
Fig E.1.The figure illustrates the observed rates of child removals and preventive services as a function of the age-corrected risk scores generated by the XGBoost model, during the four-month period following the receipt of a referral.The figure is only based on test sample data.The vertical error bars correspond to the 95% confidence intervals.

Table A .
1 -Continued from previous page Notes: In this table, we provide information on the children who were the object of a referral for maltreatment received by Danish municipalities between April 2016 and December 2017.These statistics are computed based on 173,044 referrals, representing 90,644 different children.Missing values are replaced by the median of the non-missing observations in the training sample.The unit of observation is the referrals.

Table A .
2. Summary of explanatory variables in the full information set In this table, we provide information on the children who were the object of a referral for maltreatment received by Danish municipalities between April 2016 and December 2017.These statistics are computed based on 173,044 referrals, representing 90,644 different children.Missing values are replaced by the median of the non-missing observations in the training sample.The unit of observation is the referrals.

Table B .1. Results for boys and girls
The table provides AUC scores and the associated 95% confidence interval for the four predictive models.The AUC scores for boys are based on 28,527 referrals (representing 14,904 unique children) from the test sample.The AUC scores for girls are based on 24,122 referrals (representing 12,437 unique children) from the test sample.The estimation and evaluation period starts in April 2016 and ends in December 2017.The confidence intervals for AUC-ROC are calculated by bootstrap using test sample data.

Table B .2. Results for children of non-Western and Western origin
The table provides AUC scores and the associated 95% confidence interval for the four predictive models.The AUC scores for children of Western origin are based on 42,709 referrals (representing 22,017 unique children) from the test sample.The AUC scores for children of non-Western origin are based on 9,940 referrals (representing 5,324 unique children) from the test sample.The estimation and evaluation period starts in April 2016 and ends in December 2017.The confidence intervals for AUC-ROC are calculated by bootstrap using test sample data.

Table B .3. Results for children of low and high SES
The table provides AUC scores and the associated 95% confidence interval for the four predictive models.The AUC scores for low-SES children are based on 43,187 referrals (representing 21,612 unique children) from the test sample.The AUC scores for high-SES children are based on 9,462 referrals (representing 5,747 unique children) from the test sample.The estimation and evaluation period starts in April 2016 and ends in December 2017.Standard errors were calculated by boostrap using test sample data.

Table B .4. Results for children of different ages
The table provides AUC scores and the associated 95% confidence interval for the four predictive models.The AUC scores for the 0-5-year-old children are based on 13,262 referrals (representing 7,135 unique children) from the test sample.The AUC scores for the 6-14-year-old children are based on 30,455 referrals (representing 16,229 unique children) from the test sample.The AUC scores for the 15-17-year-old children are based on 8,932 referrals (representing 4,947 unique children) from the test sample.The estimation and evaluation period starts in April 2016 and ends in December 2017.Standard errors were calculated by boostrap using test sample data.